Skip to content

Test result attestation#1203

Merged
spoorcc merged 3 commits into
mainfrom
test-result-attestation
May 14, 2026
Merged

Test result attestation#1203
spoorcc merged 3 commits into
mainfrom
test-result-attestation

Conversation

@spoorcc
Copy link
Copy Markdown
Contributor

@spoorcc spoorcc commented May 13, 2026

Summary by CodeRabbit

  • New Features

    • Releases now include cryptographic attestations for release artifacts, source archives, and test results so users can verify integrity.
  • Documentation

    • Added a how-to guide detailing verification steps with GitHub CLI across platforms.
    • Updated installation docs to reference the new verification guide.
  • Chores

    • CI/workflow verification and attestation publishing updated to support the new verification flow.

Review Change Stack

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 13, 2026

Warning

Rate limit exceeded

@spoorcc has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 17 minutes and 1 second before requesting another review.

You’ve run out of usage credits. Purchase more in the billing tab.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 9d1726a5-e7ca-4d72-9747-ed3a1202bc6d

📥 Commits

Reviewing files that changed from the base of the PR and between ad6759a and 10ddfa7.

📒 Files selected for processing (7)
  • .github/workflows/build.yml
  • .github/workflows/ci.yml
  • .github/workflows/source-provenance.yml
  • .github/workflows/test.yml
  • doc/howto/verify-integrity.rst
  • doc/index.rst
  • doc/tutorials/installation.rst

Walkthrough

This PR adds attestation publishing and verification steps to GitHub Actions (expanded job permissions, allowlist, and attestation publishing in the test job), renames the source archive to dfetch-source.tar.gz across workflows, and adds a how-to guide with platform-specific gh attestation verify examples. Installation docs now point to the new guide.

Changes

Attestation Publishing and Verification

Layer / File(s) Summary
Workflow permissions and network infrastructure
.github/workflows/ci.yml, .github/workflows/test.yml
CI test job is wired to depend on source-provenance. Both workflows request attestations: write and id-token: write. Test runner egress allowlist is extended with GitHub API/upload hosts, Sigstore endpoints (Fulcio/Rekor/TUF CDN), and *.blob.core.windows.net wildcard.
Test workflow attestation generation and publishing
.github/workflows/test.yml
pytest/behave steps now have IDs; test job conditionally downloads the canonical source-archive (non-PR), generates test-result-predicate.json from step outcomes, verifies dfetch-source.tar.gz exists, and publishes an attestation via actions/attest when appropriate.
Source archive rename and verification
.github/workflows/source-provenance.yml, .github/workflows/build.yml
Source archive filename changed from source.tar.gz to dfetch-source.tar.gz for generation, upload, attestation, and verification steps across provenance and build workflows.
Attestation verification documentation
doc/howto/verify-integrity.rst, doc/index.rst, doc/tutorials/installation.rst
New how-to documents four attestation types (SLSA provenance, CycloneDX SBOM, Verification Summary Attestation, in-toto test-result) and provides gh attestation verify examples for Linux/macOS/Windows; index and installation tutorial updated to reference the guide.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

Suggested labels

github_actions, documentation

Suggested reviewers

  • ben-edna
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Test result attestation' accurately captures the main change: the PR adds test result attestation functionality to the CI/CD workflow.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test-result-attestation

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/test.yml:
- Around line 131-137: Add a defensive pre-check before the "Attest test
results" step to verify the subject file exists (subject-path: source.tar.gz)
and fail with a clear message if missing; insert a small run step (e.g., in the
same job, before the actions/attest step) that tests for the presence of
source.tar.gz and exits non‑zero with a descriptive error so the attestation
step is only reached when the artifact file is actually present, ensuring
predicate-path: test-result-predicate.json is only consumed when source.tar.gz
exists.
- Around line 117-129: The predicate currently hardcodes "result": "PASSED" in
the "Generate test result predicate" step and must instead derive the result
explicitly from prior test step outcomes; update the workflow to add an
always()-run step (e.g., a step named "aggregate-test-status" or similar) that
inspects the relevant test steps' outcomes (use each test step's outcomes like
steps.<test-step-id>.outcome or the job status) and sets a build-level
output/variable (e.g., tests_result = "PASSED" or "FAILED") via GITHUB_OUTPUT,
then change the "Generate test result predicate" step to consume that output and
emit test-result-predicate.json with "result": "${{
steps.aggregate-test-status.outputs.tests_result }}" so the attestation reflects
explicit verification rather than a hardcoded value.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: bb7ab6e3-79fc-4d6b-a3e0-3c5d0e05e660

📥 Commits

Reviewing files that changed from the base of the PR and between addfa6e and dc7babc.

📒 Files selected for processing (5)
  • .github/workflows/ci.yml
  • .github/workflows/test.yml
  • doc/howto/verify-integrity.rst
  • doc/index.rst
  • doc/tutorials/installation.rst

Comment thread .github/workflows/test.yml
Comment thread .github/workflows/test.yml
@spoorcc spoorcc force-pushed the test-result-attestation branch from dc7babc to 1a4f354 Compare May 13, 2026 21:54
@spoorcc spoorcc force-pushed the test-result-attestation branch from 1a4f354 to ad6759a Compare May 14, 2026 07:54
@spoorcc spoorcc force-pushed the test-result-attestation branch from ad6759a to 10ddfa7 Compare May 14, 2026 08:37
@spoorcc spoorcc merged commit 2011364 into main May 14, 2026
36 checks passed
@spoorcc spoorcc deleted the test-result-attestation branch May 14, 2026 08:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant